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ABSTRACT 



This report includes comprehensive goal, objective, 
indicator, and use statement data; analysis of indicators; conclusions; and 
recommendations. In spring 1994, Florida’s Pensacola Junior College (PJC) 
undertook the second assessment of institutional effectiveness using an 
assessment plan developed between 1990 and 1993. Outcomes were examined for 
the 1993-94 academic year with respect to the 51 goals, 147 objectives, and 
147 indicators of effectiveness identified by the assessment plan in 16 
functional areas. Area supervisors were requested to report on at least one 
objective and one indicator, although many selected more than one. Results of 
the process included the following: (1) a total of 154 indicator reports were 

received from the supervisors for 1993-94; (2) based on these reports, 86.4% 

(n=133) of the indicators were fully achieved in terms of levels of 
performance designated in the assessment plan; and (3) indicators that were 
not met included achieving the correct placement of 90% of 

first- time- in-college students, implementing computerized placement tests, 
and maintaining course syllabi on file. As a result of the process, it was 
recommended that the assessment be continued in 1994-95 and simplified to 
reduce the number of indicators; that the total number of indicator reports 
generated be increased; and that more product-oriented, as opposed to 
process-oriented, indicators be developed. Data tables are appended. (HAA) 
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Introduction: In an on-going effort to improve College operations and to improve the 
documentation of institutional effectiveness, a plan of assessment was developed in 1990-1993. In 
the spring of 1993, an initial run of the Pensacola Junior College Institutional Effectiveness Plan was 
undertaken for at least a single indicator and objective for each goal in the plan. In the spring of 
1994, the second institutional effectiveness assessment was made according to this plan. Following 
are the results of the second institutional effectiveness assessment. It must be noted that this effort 
is not a final product, only the second step of an evolving process. 
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A Word of Caution: These indicators are intended to be data for affirmation of the mission of the 
college and for continued improvement of the institution. The success of the process requires a non- 
threatening environment in which the aggressive pursuit of evaluative data, both positive and 
negative, is encouraged. Consequently the achievement or non-achievement of a given indicator 
standard, while extremely important to the operational unit, is not as important to this purpose as 
is the fact that the college has examined that facet of the college's productivity, has generated 



discrete information concerning it, and has proceeded 
of the colleee. 



to act on that information for the improvement 



3 



1 



EXECUTIVE SUMMARY 



The second institutional effectiveness assessment was undertaken by the Office of Institutional 
Research and Effectiveness in the spring of 1994, for the 1993-1994 academic year. This used the 
plan developed by the Institution Effectiveness Taskforce between 1990 and 1993, which consisted 
of sixteen functional areas, 5 1 goals, 147 objectives and 147 possible indicators. For the second run, 
area supervisors were requested to select one objective and one indicator for each of the goals, 
although some selected more. Reports on performance indicators for these objectives were received 
through August 1, 1994. A follow-up process was made through October 10. The process was 
successfully competed this year for the second time. 

Findings: 

(1) A total of 154 indicator reports were received for 1993-1994 (some indicator standards were 
used by more than one unit.) 

(2) Of these, 1 33 or 86.4% were fully met in terms of designated levels of performance. 

(3) Twenty-one did not meet the specified standards. 

(4) Data from these assessments were used to a high degree (98.7%). 

Conclusions: 

The central mission of the college is affirmed by the results of this assessment, unit-bv-unit 
across the college. 

Recommendations: 

(1) The process must be continued in 1994-1995. 

(2) Continued simplification of the process is in order; especially with the reduction of the 
number of indicators. 

(3) Each institutional unit should strive to focus more of its institutional effectiveness efforts 
upon indicators that show basic college productivity respective of its mission, rather than 
process activity measures. Although processes of the college are important they represent 
means toward ends rather than outcome productivity. 



ANALYSIS 



1 . It is given under the current plan that each functional area (n= 1 6) can have multiple goals (n=5 1 ), 
which in turn may have multiple objectives (n=147) and multiple indicators (n=253). In addition 
there is the potential for multiple reports based on individual indicators. However in 1993-1994 
assessment, there was to be an indicator reported for each goal; that is a minimum 5 1 indicator 
responses. There were in fact 90 individual indicators actually selected to be reported. Some 
received multiple responses for the same objective, some were single responses for a total of 154 
indicator reports in all. Thus the indicator reporting rate was 35.6% for 1993-1994 (that is the 
percentage of individual standards reported at least one time. A survey of these reports is 
presented in TABLE I. 

2. TABLE II gives detailed information on these data. Of the 154 total indicator standard reports 
received, 133 or 86.4% of the prescribed standards were met or exceeded. Of all the reports 
received, the reporters stated in 1 52 or 98.2% of these instances that the report data (whether the 
standard was achieved or not) was used by the reporting unit for its improvement. While the 
majority of data dealt with “process” indicators, 14 or 9.1% were output or product indicators 
(of which 10 or 71.4% were fully achieved). 

3. TABLE III displays current “output” or mission productivity indicators. The output indicators 
address mission end products rather than the processes that help produce these outputs. 

4. TABLE IV displays indicators that have yet to be examined either in 1 992-1993 or 1 993-1994, 
but which must be assessed at least once during the five-year cycle of evaluation. 
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CONCLUSIONS 



It is important to recognize that this evaluation process is intended to assess the assessment process 

itself as well as to assess college performance this year. 

1 . Overall indicator response suggests that PJC units are meeting or exceeding prescribed 
performance levels for the objectives assessed: and as a result, are meeting, college mission goals . 

2. The pilot trial run using the current procedures for data collection was a good one. This second 
run was adequate. The level of cooperation by the participating reporters was excellent. The 
reporting system and paper trail worked well and had a minimal impact on college resources. 

3. Too many indicators were found to be process rather than product oriented. More product 
outcome measures are needed. 

4. An overreliance on satisfaction surveys was noted. There should be a curtailment of survey-based 
indicators and more emphasis placed on outcome measures of a quantitative nature. 

5. Subtle changes in some objectives and indicators continued to be made in the course of the 
process. This resulted in some indicator objectives having evolved into objectives more 
appropriate to unit operational plans or institution strategic plan objectives. 

6. Little opposition to the process trial run and this second run was noted, though lack of full 
understanding of the process or its purpose was frequently encountered. (Some division of opinion 
seems to exist between those who see the process as a mirror of SACS Criteria and those who 
envision it as a test of college mission compliance.) Some see the process as an added burden. 
Others confuse it with Strategic Planning and the SACS reaffirmation processes. 



That: 



RECOMMENDATIONS 



1 . The process be continued, simplified and refined. 

2. College leadership reaffirm the importance of the process. 

3. In 1995-1996, the total number of indicator reports must be increased, with special emphasis on 
these indicators that relate to outcome productivity measures rather than measures of internal 
processes (see TABLE III & TABLE IV). 

4. Individual reporting units continue to utilize the results of these assessments for continuous 
improvement. 

5. Continued efforts must be made to educate PJC personnel on the institutional effectiveness 
process. 
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70% of students are satisfied with student information services 
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TABLE II. 



ANALYSIS OF REPORTED INSTITUTIONAL EFFECTIVENESS INDICATORS (a) 

1993-1994 
(YEAR II.) 



TOTAL INDICATOR REPORTS RECEIVED 


154 


100.0% 


PRODUCTIVITY INDICATORS REPORT RECEIVED 


14 


9.1% 


PRODUCTIVITY INDICATORS REPORTS ACHIEVED 


71.4% 


10 


6.5% 


PROCESS INDICATOR REPORTS RECEIVED 


140 


90.9% 


PROCESS INDICATORS ACHIEVED 


87.9% 


123 


79.9% 


ALL INDICATORS ACHIEVED 


133 


86.4% 


ALL INDICATORS DATA USED 


152 


98.7% 



(a) Source: Tabulated from TABLE I. data. 
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TABLE m 



MASTER LIST OF PRODUCT OUTPUT INDICATORS 

These are the indicators that exist as of 1 1 -22-94 that measure PJC mission productivity. 
An asterisk (*) indicates which of these were assessed in the 1993-1994 process. 

ADMISSIONS I. 

I.B.l.a. I.B.3.b. 

I.B.2.a. I.B.4.a. 

I.B.3.a. I.C.l.a. 

COMPLETION REQUIREMENTS II. 

* II.A.2.a. II.B.2.a. 

* II.B.l.a. 

* II.B.l.b. 

CURRICULUM ni. 



IDA. l.a. 


III.A.2.a. 


* I II.B.l.a. 


* IU.B.l.e. 


III. A. Lb. 


III.A.2.b. 


in.B.l.b. 


III.B.2.a. 


III.A.1.C. 


III.A.2.d. 


III.B.l.c. 


III.B.2.b. 


IILA.l.e. 


III.A.2.e. 


III.B.l.d. 


III.B.2.d. 








III.B.2.e 



COMMUNITY PROGRAMS IV. 



IV .A. La. 


IV.A.3.C. 


IV.B.3.a. 


* IV.C.2.U. 


IV.A.2.a. 


IV.A.3.d. 


IV.B.3.b. 


IV.C.3.a. 


IV.A.2.b. 


IV.A.3.e. 


IV.B.3.C. 


IV.C.4.a. 


IV.A.2.C. 


IV.B.l.a. 


IV.B.3.d. 


IV.C.4.b. 


IV.A.3.a. 


* IV.B.l.b. 


IV.B.3.e. 


IV.D.l.a. 


IV.A.3.b. 


IV.B.2.a. 


IV.C.l.a. 


IV.D.l.b. 



FACULTY V. 
None. 
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LEARNING RESOURCES VI. 

VI.A.5.C. 

VI. B.2.a. 

* VLC.l.a. 

INSTRUCTIONAL SUPPORT VII. 

VII. A.l.a. VII.B.l.a. 

VII.A.l.b. VII.B.l.b. 

COMPUTER SERVICES VIII. 

None. 

ORGANIZATION & ADMINISTRATION IX. 
IX.A.5.a. 

FINANCIAL RESOURCES X. 

None. 

PHYSICAL RESOURCES XI. 

None. 

GRANTS XII. 

XU.B.l.a. 

INSTRUCTION XHI. 



XIIl.AAa. 


XIII.A.5.b. . 


* XIlI.C.2.a. 


XIII.C.2.d. 


XIII.A.4.b. 


XIII.A.7.a. 


XIII.C.2.b. 


XIII.C.3.a. 


XIII.A.4.C. 


XIII.B.l.c. 


XIII.C.2.a. 


XIII.C.3.b. 








XIII.C.4.a. 








xm.c.4.b. 



INSTITUTIONAL ADVANCEMENT XIV. 
None. 
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INSTITUTIONAL RESEARCH XV. 
None. 



STUDENT DEVELOPMENT SERVICES XVI. 



* XVI. A. l.a. 

* XVI.C.3.C. 
XVI.C.4.a. 



XVI.C.4.C. 

XVI.C.5.a. 

XVI.C.5.b. 

XVI.F.l.c. 



XVI.F.l.a. 

XVI.F.l.b. 

XVI.F.l.c. 

XVI.H.3.a. 



XVI.G.l.b. 

* XVI.H.l.a. 
XVI.H.3.a. 

t XVI.I.l.a. 

* XVI.I.2.a. 
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TABLE IV. 

LIST OF UNREPORTED INDICATORS THROUGH 1993-1994 



New I.A.l.a. 


III.A.2.b. 


IV.C.4.b. 


XII.B.2.a. 


XIII.C.4.b. 


New I. A. l.b. 


III.A.2.C. 


IV.D.l.a. 


XIII. A. La. 


XTV.C.2.a. 


New I.A.l.c. 


III.A.2.d. 


IV.D.l.b. 


xm.A.i.b. 


XIX.C.3.b. 


New I.A.l.d. 


III.A.2.e. 


IV.D.2.a. 


XIII. A. l.c. 


XIV.C.4.a. 


New I.A.2.a. 


III.B.l.c. 


IV.D.3.a. 


XIII.A.3.a. 


XV.A.2.a. 


New I.A.2.b. 


III.B.l.d. 


IV.D.3.b. 


XIII.A.3.b. 


XV.A.2.b. 


New I.A.3.a. 


III.B.2.a. 


V.A.l.a. 


Xni.A.4.a_ 


XV.A.3.a. 


New I.A.3.b. 


III.B.2.b. 


V.A.3.a. 


X3II.A.4.b. 


XV.A.5.b. 


New I.C.l.a. 


III.B.2.C. 


V.B.l.a. (1) 


X3II.A.4.C. 


XVI. C. La. 


New I.E.2.a. 


III.B.2.d. 


V.B.l.a. (2) 


XIII.A.S.a. 


XVI. C. Lb. 


New IV.B.l.b. 


III.B.2.e. 


V.B.l.a. (3) 


Xni.A.5.b. 


XVI.C.2.a. 


New XVI.H.2.b. 


IV.A.2.a. 


V.C.l.a. 


XIII.A.6.a. 


XVI.CAa. 


NewXVI.B.2.b. 


IV.A.2.b. 


V.C.l.c. 


XHI.A.6.b. 


XVI.C.4.b. 


I.B.La. 


IV.A.2.C. 


V.C.l.d. 


Xm.A.7.a. 


XVI.C.4.C. 


I.B.2.a. 


IV.A.3.a. 


V.C.l.e. 


XIII.A.7.b. 


XVI.C.S.a. 


I.B.3^. 


IV.A.3.b. 


V.C.2.a. 


XIII.A.7.C. 


XVI.C.S.b. 


I.B:3.b. 


IV.A.3.C. 


V.C.4.a. 


X3II.B.1.C. 


XVI.C.6.a. 


I.B.4.a. 


IV.A.3.d. 


VI.A.5.a. 


Xin.B.2.a. 


XVI.D.l.a. 


I.C.l.b. 


IV.A.3.e. 


VI.B.3.a. 


Xm.B.2.c. 


XVI'D.l.b. 


I.C.2.a. 


IV.B.l.a. 


VILA. La. 


XUI.C.l.b. 


XVI.D:3.a. 


I.D.l.b. 


IV.B.3.a. 


VII.B.l.a. 


XUI.C.l.c. 


XVI.E.l.b. 


II. A. l.a. 


IV.B.3.b. 


vn.B.r.b. 


XIII.C.2.a. 


XVI.E.2.a. 


II.B.2.a. 


IV.B.3.C. 


VIII.D.2.a. 


XIII.C.2.b. 


XVI.E.2.b. 


III. A. Lb. 


IV.B.3.d. 


IX.A.3.a. 


X3II.C.2.C. 


XVI.E.3.a. 


IILA.l.c. 


IV.B.3.e. 


IX.A.5.a. 


XIII.C.2.d. 


XVI.E.3.b. 


III.A.Ld. 


IV.C.l.a. 


XI.B.l.a. 


XIII.C.3.a. 


XVI.E.3.C. 


III.A.l.e. 


IV.C.3.a. 


XI.C.3.a. 


XIII.C.3.b. 


XVI.F.l.b. 


III.A.2.a. 


IV.C.4.a. 


XII.A.2.C. 


XIII.C.4.a. 


XVI.F.l.c. 
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XVI. F. 2. a. 

XVI.F.2.b. 

XVI.G.l.a. 

XVI. G. l.b. 

XVI.G.2.a. 

XVI.H.3.a. 
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